Meta Soft Label Generation for Noisy Labels

The existence of noisy labels in the dataset causes significant performance degradation for deep neural networks (DNNs). To address this problem, we propose a Meta Soft Label Generation algorithm called MSLG, which can jointly generate soft labels using meta-learning techniques and learn DNN parameters in an end-to-end fashion. Our approach adapts the meta-learning paradigm to estimate optimal label distribution by checking gradient directions on both noisy training data and noise-free meta-data. In order to iteratively update soft labels, meta-gradient descent step is performed on estimated labels, which would minimize the loss of noise-free meta samples. In each iteration, the base classifier is trained on estimated meta labels. MSLG is model-agnostic and can be added on top of any existing model at hand with ease. We performed extensive experiments on CIFAR10, Clothing1M and Food101N datasets. Results show that our approach outperforms other state-of-the-art methods by a large margin.Comment: Accepted by ICPR 202

MetaLabelNet: Learning to Generate Soft-Labels from Noisy-Labels

Real-world datasets commonly have noisy labels, which negatively affects the performance of deep neural networks (DNNs). In order to address this problem, we propose a label noise robust learning algorithm, in which the base classifier is trained on soft-labels that are produced according to a meta-objective. In each iteration, before conventional training, the meta-objective reshapes the loss function by changing soft-labels, so that resulting gradient updates would lead to model parameters with minimum loss on meta-data. Soft-labels are generated from extracted features of data instances, and the mapping function is learned by a single layer perceptron (SLP) network, which is called MetaLabelNet. Following, base classifier is trained by using these generated soft-labels. These iterations are repeated for each batch of training data. Our algorithm uses a small amount of clean data as meta-data, which can be obtained effortlessly for many cases. We perform extensive experiments on benchmark datasets with both synthetic and real-world noises. Results show that our approach outperforms existing baselines

Image Classification with Deep Learning in the Presence of Noisy Labels: A Survey

Image classification systems recently made a giant leap with the advancement of deep neural networks. However, these systems require an excessive amount of labeled data to be adequately trained. Gathering a correctly annotated dataset is not always feasible due to several factors, such as the expensiveness of the labeling process or difficulty of correctly classifying data, even for the experts. Because of these practical challenges, label noise is a common problem in real-world datasets, and numerous methods to train deep neural networks with label noise are proposed in the literature. Although deep neural networks are known to be relatively robust to label noise, their tendency to overfit data makes them vulnerable to memorizing even random noise. Therefore, it is crucial to consider the existence of label noise and develop counter algorithms to fade away its adverse effects to train deep neural networks efficiently. Even though an extensive survey of machine learning techniques under label noise exists, the literature lacks a comprehensive survey of methodologies centered explicitly around deep learning in the presence of noisy labels. This paper aims to present these algorithms while categorizing them into one of the two subgroups: noise model based and noise model free methods. Algorithms in the first group aim to estimate the noise structure and use this information to avoid the adverse effects of noisy labels. Differently, methods in the second group try to come up with inherently noise robust algorithms by using approaches like robust losses, regularizers or other learning paradigms

Kirli Etiketlerin Varlığında Derin Öğrenme: Meta-Öğrenim Yaklaşımı

Image classification systems recently made a giant leap with the advancement of deep neural networks. However, these systems require an excessive amount of labeled data to be adequately trained. Gathering a correctly annotated dataset is not always feasible due to practical challenges. Because of these practical challenges, label noise is a common problem in real-world datasets. This thesis presents two novel label noise robust learning algorithms: MSLG (Meta Soft Label Generation) and MetaLabelNet. Both algorithms are powered by meta-learning techniques and share the same learning framework. Proposed algorithms generate soft labels for each instance according to a meta-objective, which is to minimize the loss on the small meta-data. Afterward, the main classifier is trained on these generated soft-labels instead of given noisy labels. In each iteration, before conventional learning, the proposed meta objective reshapes the loss function so that resulting gradient updates would lead to model parameters with the minimum loss on meta-data. Different from MSLG, MetaLabelNet can work with dataset consists of both noisily labeled and unlabeled data, which is a problem setup that is not considered in the literature up to now. To prove the validity of the proposed algorithms, they are backed with mathematical justification. Extensive experiments on datasets with both synthetic and real-world label noises show the superiority of the proposed algorithms. For comparison with the state-of-the-art methods, proposed algorithms are tested on widely used noisily labeled benchmarking dataset Clothing1M. Both algorithms beat the baseline methods with a large margin, where MSLG achieves 2.3\% and MetaLabelNet achieves 4.2\% higher than the closest method. Results show that presented approaches are fully implementable for real-world use cases. Additionally, a novel label noise generation algorithm is presented for the purpose of generating realistic synthetic label noise.Derin öğrenme algoritmalarının gelişmesi ile günümüzde bilgisayarlı görü teknolojilerinde büyük bir sıçrama yaşanmaktadır. Ancak yapay sinir ağlarını eğitmek için yüksek miktarda etiketlenmiş veri gerekmektedir. Veri setlerinin tamamıyla doğru etiketlenmesi çoğu zaman mümkün olmamaktadır. Bu tezde iki adet etiket kirliliğine karşı dayanıklı öğrenme algoritması önerilmiştir: MSLG (Meta Yumuşak Etiket Öğrenimi) ve MetaLabelNet. İki algoritma da meta-öğrenme tekniklerinden faydalanmakta ve aynı öğrenme sisteminden istifade etmektedir. Önerilen algoritmalar meta hedefe göre meta veri üzerinde kaybı en aza indirecek şekilde yumuşak etiketler üretir. Sonrasında ana sınıflandırıcı model kirli etieketler yerine üretilen yumuşak etiketler üzerinde eğitilir. Her eğitim adımında geleneksel makine öğreniminden önce, meta hedef model parametrelerinin en az kirlilikten etkilenecek şekilde güncellenmesine sebep olacak etiketleri üretir. Ayrıca MetaLabelNet hem kirli hem de etiketsiz verinin oluşturduğu veri setleri üzerinde de çalışabilmektedir. Bu problem türü literatürde daha önce çalışılmamıştır. Önerilen algoritmaların geçerliliği matematiksel olarak da kanıtlanmıştır. Metotların performansı hem sentetik hem de dünya kaynaklı gerçek kirliliğe sahip veri setleri üzerinde test edilmiştir. Sonuçlar önerilen algoritmaların yüksek başarısını doğrulamaktadır. Literatürdeki güncel algoritmalar ile performans kıyaslaması yapabilmek adına, önerilen algoritmalar bu alanda yaygın olarak kullanılan Clothing1M veri seti üzerinde test edilmiştir. İki algoritma da literatürdeki diğer metotların başarısının üstüne çıkmıştır. MSLG en başarılı metottan 2.3\% fazla performans sağlarken MetaLabelNet 4.2\% yüksek performans elde etmiştir. Sonuçlar algoritmaların gerçek dünya uygulamalarında kullanıma hazır olduğunu göstermektedir. Ayrıca tez kapsamında gerçekçi etiket kirliliği oluşturmak maksatlı da özgün bir algoritma önerilmiştir.Ph.D. - Doctoral Progra